Controlling an adaptation of a behavior of an audio device to a current acoustic environmental condition

ABSTRACT

It is described a method for controlling an adaptation of a behavior of an audio device ( 100 ) to a current acoustic environmental condition. The method comprises (a) monitoring an audio output signal (x(t), x′(t)) being provided to an acoustic output device ( 110 ) of the audio device ( 100 ) for outputting an acoustic output signal, (b) measuring an audio input signal (z(t)) being provided by an acoustic input device ( 120 ) of the audio device ( 100 ), wherein the audio input signal (z(t)) is indicative for a feedback portion of the acoustic output signal and for the current acoustic environmental condition, (c) determining a relation between the audio output signal (x′(t)) and the audio input signal (z(t)) and (d) adapting the behavior of the audio device ( 100 ) based on the determined relation. Further, it is described a data processor, a computer-readable medium and an audio device, which are adapted to control and/or to carry out the above mentioned method for controlling an adaptation of the behavior of an audio device ( 100 ) to a current acoustic environmental condition.

FIELD OF THE INVENTION

The present invention relates to the technical field of controllingaudio systems, which comprise both an acoustic output device such as aloudspeaker and an acoustic input device such as a microphone. Inparticular, the present invention relates to a method for controlling anadaptation of an audio output signal of an audio device to a currentacoustic environmental condition of the audio device. Further, thepresent invention relates to a data processor, to a computer-readablemedium and to an audio device, which are adapted to control and/or tocarry out the above mentioned method for controlling an adaptation of anaudio output signal to a current acoustic environmental condition.

BACKGROUND OF THE INVENTION

Depending on the situation in which a mobile device such as for instancea mobile phone is used, the desired level of an audio playback signal orof a ringtone indicating an incoming call or an incoming Short MessageService (SMS) needs to be different. For example, when in a meeting orin a rather silent office or home condition with other people in theroom, one would like the playback volume to be rather low in order notto disturb other people. On the other hand, when being in a noisyenvironment such as a car, a pub, or in the street, one would like theringtone to be loud enough so that the alert of an incoming call isalways audible. Furthermore, when the mobile device is covered or keptin a closed environment like a pocket or bag, the ringtone isacoustically attenuated and is likely not to be heard at all even in lownoisy circumstances.

In most mobile devices the playback volume of the ringtone can beadjusted manually, typically as a setting in the mobile phone's menusystem. Alternatively, the ringtone volume might be controlled byso-called profiles. Thereby, the user can manually switch from oneprofile to another in order to change the ringtone volume. In addition,most mobiles allow the ringtone volume to increase over time, startingfrom a very soft the moment the call comes in, to very loud after afixed period of time. However, manual ringtone adjustment or profileswitching has the disadvantage that it requires user interaction,something people tend to forget, resulting in undesirable phonebehavior. An automatic volume increase is not ideal either, because whenone is located in a noisy environment only the very last part of theringtone will be loud enough. However, because it will be audible onlyfor a very short period, chances are high that the alert is not heard bythe user and unintentionally the voice mail system will answer theincoming call.

Further, in most mobile devices a vibration feature can be enabled sothat the alert of an incoming call can also be felt when keeping themobile in a pocket in close contact with the body. However, some peopledo not want to have the vibration function enabled at all times or theysimply tend to forget to enable it because it requires the user'sattention and manual interaction. Furthermore, when the mobile is keptin a pocket or bag, there is no immediate contact with the user's bodyso that the vibration function will not help in notifying an alertrespectively an incoming call.

More advanced mobile phones have built-in sensors like an ambient lightsensor, a proximity sensor, an accelerometer, etc. Such sensors can beused to learn something about the environment of the mobile phone.However the information extracted using these sensors is typically notconclusive. For example, the ambient light sensor can be used to detectwhether or not the mobile phone is covered or is located in a pocket orbag. However, it cannot distinguish this situation from the situationwhere the mobile phone is lying on a night table in the dark. In thefirst situation one would like to increase the loudness of the ringtoneto compensate for the fact that it will be acoustically attenuated,whereas in the second situation one would like the ringtone playback tobe gentle.

It has been proposed to measure the environmental noise level using thebuilt-in microphone of the mobile device during a short period of timebefore starting an audio playback signal or a ringtone. However, thiscould lead to practical robustness issues as the level of an audioplayback signal or ringtone is only based on the noise level estimationbefore the audio playback or the ringtone becomes effective. It is notpossible to deal with variations in the environmental noise after theaudio playback signal or the ringtone has started.

It has been further proposed to measure the noise level during audioplayback signal or ringtone. However, due to a typically high acousticcoupling between the speaker and the microphone of a mobile device, acontinuous echo of the audio playback signal or the ringtone will bedominant with respect to the captured ambient noise. A direct noiselevel measurement based on the microphone signal will lead to incorrectnoise estimations, resulting in an incorrect audio playback signal orringtone adjustment.

Furthermore, an acoustic echo canceller has been proposed, which iscapable of removing the audio playback signal or the ringtone from themicrophone signal. However, due to a typically very high acousticcoupling between the speaker and microphone on a mobile device, anysmall mismatch between the estimated echo and the echo captured by themicrophone affects the quality of the remaining microphone signal.Hence, by removing the audio playback echo or the ringtone echo, theacoustic echo canceller degrades the remaining residual signal and hencealso the captured ambient noise. This leads to an incorrect noise levelestimation.

As elucidated above, requirements with respect to audio playback orringtone are dependent on the situation in which the mobile device isbeing used and known procedures for measuring the noise of the ambientenvironment suffer from a high acoustic coupling between the speaker ofthe mobile device and the microphone of the mobile device. Therefore,there may be a need for automatically adapting the volume of an audioplayback signal or a ringtone depending on the current ambientconditions in an appropriate, easy and effective manner.

OBJECT AND SUMMARY OF THE INVENTION

This need may be met by the subject matter according to the independentclaims. Advantageous embodiments of the present invention are describedby the dependent claims.

According to a first aspect of the invention there is provided a methodfor controlling an adaptation of a behavior of an audio device to acurrent acoustic environmental condition. The provided method comprises(a) monitoring an audio output signal being provided to an acousticoutput device of the audio device for outputting an acoustic outputsignal, (b) measuring an audio input signal being provided by anacoustic input device of the audio device, wherein the audio inputsignal is indicative for a feedback portion of the acoustic outputsignal and for the current acoustic environmental condition, (c)determining a relation between the audio output signal and the audioinput signal and (d) adapting the behavior of the audio device based onthe determined relation.

This first aspect of the invention is based on the idea that anenvironmental background noise has a strong impact on the relationbetween the audio output signal and the captured audio input signal.Therefore, by measuring and/or monitoring this relation importantinformation about the acoustic environment of the audio device can beextracted. In particular, compared to a silent surrounding a significantenvironmental background noise may disturb this relation. In accordancewith the described method the measurement and a subsequent analysis ofthis disturbance may be used to steer the adaptation of the audio outputsignal to a clear audible level with respect to the environmentalbackground noise. Thereby, an ambient-aware adaption of the audio outputsignal can be realized.

It has to be mentioned that according to the invention it is notnecessary, however not forbidden, to directly measure the ambientrespectively the environmental noise. Such a direct measurement could becarried out either before or during the described audio output signaladaptation method.

Generally speaking, the relation between the audio output signal and theaudio input signal reflects the acoustical characteristics of the audiodevice and of the environment of the audio device, which may compriseboth the acoustic output device and the acoustic input device.Monitoring the dynamics or changes and disturbances in this relationwith respect to reference situations (e.g. no background noise, audiodevice lying freely on a table), may reveal information about theacoustic environment of the audio device and changes in this acousticenvironment.

The described feedback portion of the acoustic output signal may begiven by an acoustic coupling between the acoustic output device and theacoustic input device. Thereby, at least a portion of the acousticoutput signal is fed back from the acoustic output device to theacoustic input device. Of course this portion strongly depends on thecorresponding acoustic path, which may be characterized by certainattenuation and/or a certain modification of the frequency distributionof the fed back acoustic output signal.

The terms “audio output signal” and “audio input signal” refer tonon-acoustical signals. In particular, the term “audio output signal”may refer to an electrical signal which is provided to the acousticoutput device in order to be transformed into the acoustic output signal(i.e. a sound wave). Correspondingly, the term “audio input signal” mayrefer to an electrical signal which is produced by the acoustic inputdevice in response to the receipt of the acoustic input signal and/orenvironmental background sound signals, which are also sound waves.

It has to be mentioned that the described relation between the audiooutput signal and the audio input signal can also be determined by usingderivative signals of the audio output signal and/or the audio inputsignal. Depending on the relation between the audio output signal or theaudio input signal and the respective derivative signal, the relationinvolving at least one derivative signal will differ in a known mannersuch as for instance a certain factor from the direct relation betweenthe audio output signal and the audio input signal.

It is mentioned that the term “determining” has to be understood in awide manner. Determining may mean for instance estimating (in particularwhen there is no exact value for the ratio), measuring or calculating.

The behavior of the audio device may be any functionality of the audiodevice, which might be introduced, removed or modified based on thedetermined relation between the audio output signal and the audio inputsignal. Thereby, the behavior adaption may be carried out when thedetermined relation (a) reaches a predefined value, (b) changes by apredefined difference, (c) exhibits a certain dynamic change and/or (d)shows a certain disturbance with respect to a reference value.

According to an embodiment of the invention the behavior of the audiodevice is given by an amplitude and/or a frequency of the audio outputsignal, an amplitude and/or a frequency of a vibrating mechanism of theaudio device and/or a modification of the operation of a display of theaudio device. The modification of the display may comprise for instancea deactivation, an activation, an enlightening of a dimming.

According to a further embodiment of the invention the relation betweenthe audio output signal and the audio input signal is determined byapplying a cross-correlation procedure, an adaptive filtering procedureand/or a coherence estimation procedure. This may provide the advantagethat well-known procedures for relating different signals with eachother can be employed. Of course, also other non-mentioned proceduresmight be used for the determination of the described relation. Forinstance the adaptive filtering procedure may be carried out by means ofan acoustic echo canceller adaptive filter such as for instance anormalized least mean square adaptive filter.

According to a further embodiment of the invention the acoustic outputdevice is a loudspeaker and/or the acoustic input device is amicrophone. This may provide the advantage that the described audiooutput signal adaptation method can be carried out with many differenttypes of audio devices. Thereby, it is not possible that the audiodevice itself comprises the loudspeaker and/or the microphone. Thedescribed method can also be applied if the respective audio devicecomprises at least interfaces for directly or indirectly connecting theloudspeaker and/or the microphone to the audio device.

The described method may exhibit the most important advantages overprior art audio output adaptation control methods if there is a strongacoustic coupling between the loudspeaker and the microphone. In thiscase direct noise level measurements from the captured microphone signalare mostly not possible because the ambient noise is masked by thefeedback portion of the acoustic output signal.

The audio output signal may be any signal, which can be converted by theloudspeaker into sound waves. In particular, the acoustic output signalmay be an audio playback signal or an alarm signal. Thereby, anambient-aware music playback or an ambient-aware alerting of a user maybe realized. In this context, if the audio device, on which thedescribed method is carried out, is for instance a mobile phone, thealarm signal may be a ringtone indicating the user of the mobile phonean incoming call and/or an incoming SMS.

According to a further embodiment of the invention the method furthercomprises comparing the determined relation between the audio outputsignal and the audio input signal with at least one reference relation.Thereby, adapting the behavior of the audio device further takes intoaccount a result of the comparison between the determined relation andthe reference relation.

This may provide the advantage that the determined relation can beassigned to or classified into different groups of relations. Dependingon the respective group different measures for adapting the behavior ofthe audio device can be carried out.

This further embodiment allows solving the so called “closedenvironments” problem, because the determined relation between the audiooutput signal and the audio input signal reflects the acousticalcharacteristics of the audio device within its acoustic environment.When the audio device is in a closed environment like a pocket or a bagor when it is covered by soft or hard material, the acoustical couplingbetween the acoustic output device respectively the loudspeaker and theacoustic input device respectively the microphone is different. Thereby,the acoustical coupling will be lower or higher or will be continuouslychanging due to movements in the pocket or bag compared to when theaudio device is freely lying for instance on a table. This situationdependent deviation can be detected by comparing the acoustical couplingmeasure given by the comparison of the determined relation with areference situation relation. The situation dependent deviation can beused for adjusting the audio output signal in order to increase theperceived loudness.

According to a further embodiment of the invention the method furthercomprises comparing the determined relation between the audio outputsignal and the audio input signal with a threshold. Thereby, if thedetermined relation is larger or smaller than the threshold, adaptingthe behavior of the audio device comprises increasing the signal levelof the audio output signal.

It has to be mentioned that the accomplishment of the described signallevel increase may be made dependent whether the amplitude of theinitial audio output exceeds a further threshold, which can also bedenominated a volume threshold.

In case of significant ambient noise, this determined relation will bedisturbed by the captured ambient noise. Hence, by monitoring thedynamics in or disturbances on the determined relation for audio outputsignal levels, which are larger than the further threshold (volumethreshold), a significant disturbance beyond the threshold indicatesthat the noise component is dominant and that the audio output signal isnot loud enough. As long as disturbances beyond the threshold aremeasured, the audio output signal needs to be enhanced further.

A proper choice of the second (volume) threshold and/or the first(disturbance) threshold may depend on the concrete acoustical couplingcharacteristics of the audio device. Therefore, the threshold and/or thefurther threshold may need to be tuned to the acoustical characteristicsof the audio device in order to provide for an optimal audio outputsignal adaptation. The mapping of the dynamic range of the determinedrelation, changes or disturbance on this relation onto the how and theamount of adaptation may need to be tuned to the acousticalcharacteristics of the audio device.

This embodiment of the invention may provide the advantage that indirectnoise measurements are made possible even for an audio device having astrong acoustical coupling between the acoustic output device and theacoustic input device. This holds also for very silent environmentalconditions.

In case of silent environmental conditions, the determined relationbetween the audio output signal and the audio input signal reflects theacoustical characteristics of the audio device in its silentenvironment. Thereby, the audio input signal, which is captured by theaudio input device, represents a signal mix caused by the environmentalbackground noise and the feedback portion of the acoustic output signal.

According to a further embodiment of the invention the audio device is amobile communication end device. The communication end device may becapable of connecting with an arbitrary telecommunication network accesspoint such as for instance a base station. The communication end devicemay be a cellular mobile phone, a Personal Digital Assistant (PDA), anotebook computer and/or any other movable communication device.

The described method may provide the advantage that an ambient-awareringtone can indicate an incoming call. Thereby, the adaptation and inparticular an increase of the loudness or a decrease of the loudness ofthe ringtone may depend on the acoustical characteristics of theenvironment of the mobile communication end device.

In this respect it is mentioned that the ringtone may comprise anyarbitrary sound like a harmonic music, an identifiable noise or anysequence of tones having any arbitrary tone color.

According to a further embodiment of the invention the method furthercomprises detecting a picking up of the mobile communication end devicebased on a change in the determined relation between the audio outputsignal and the audio input signal. This may provide the advantage thatwhen answering the mobile phone it can be immediately detected when theuser grabs the mobile communication end device. Thereby, the detectionmay rely on a rapid change of the acoustic coupling between theloudspeaker and the microphone of the mobile phone when the user putshis hand around the mobile phone or when the user moves the mobile phonefrom its initial location.

Generally speaking, the described method provides a technique foracoustically detecting a picking up of the mobile phone for answeringincoming calls based on monitoring the acoustic coupling given by thedetermined relation between the audio output signal and the audio inputsignal. Thereby, the mobile phone can be steered to adapt in particularthe loudness of the ringtone towards a desired behavior.

According to a further embodiment of the invention the method furthercomprises generating a sensor signal by a sensor device. Thereby,adapting the audio output signal further takes into account the sensorsignal.

The described sensor device may comprise any context sensor, which iscapable of detecting a measurable variable of the audio device and/or ofthe environment of the audio device. The additional consideration of thesensor signal may provide the advantage, that the audio output signalcan be adapted very precisely towards its desired behavior depending onthe environmental acoustic conditions.

Generally speaking, the adaptation of the audio output signal can befurther enhanced by using the acoustical detection in combination withat least one sensor signal being provided by at least one other contextsensor. Thereby, additional information about the environment of theaudio device might be extracted in order to make the adaptation of theaudio output signal even more reliable.

According to a further embodiment of the invention the sensor devicecomprises a light sensitive sensor, a motion sensor, an accelerationsensor and/or a proximity sensor. At least one of such sensors, whichmay be built-in sensors of advanced mobile phones, can be used to learnsomething more specific about the environment of the audio device.

In this respect it is mentioned that the information extractedexclusively from one of such sensors is typically not conclusive. Forexample, an ambient light sensor can be used to detect whether or notthe audio device is covered or is located in a pocket or bag. However,the ambient light sensor cannot distinguish this situation from thesituation wherein the audio device is lying on a night table in thedark. In the first situation one would like to increase the loudness ofthe acoustic output signal to compensate for the fact that it will bemuffled, whereas in the second situation one would like the acousticoutput signal to be gentle. However, when additionally taking intoaccount the determined relation between the audio output signal and theaudio input signal, it may be possible to reliably distinguish betweenthe described first situation and the second situation.

It is explicitly pointed out that two or even more sensor signals beinggenerated by two or even more sensors can be taken into account foradapting the audio output signal. Preferably, these sensors are ofdifferent nature such that different types of information regarding theaudio device environment and/or the operational state of the audiodevice can be used for adapting the audio output signal in a reliablemanner. Thereby, the term “operational state” also includes the state ofmotion and/or the state of acceleration of the audio device.

According to a further aspect of the invention there is provided a dataprocessor for controlling an adaptation of a behavior of an audio deviceto a current acoustic environmental condition of the audio device.Thereby, the data processor is adapted for performing the method inaccordance with any one of the above-described embodiments.

Also this further aspect of the invention is based on the idea that bymeasuring and/or monitoring the relation between the audio output signaland the captured audio input signal valuable information about theacoustic environment of the audio device can be extracted. Thisinformation can be taken into account for optimally adapting thebehavior of the audio device and in particular the signal level of theaudio output device towards a desired and user comfortable level.

The described data processor may be a part of a system for acousticallydetecting the changes in the acoustic environment of the audio device inorder to steer the audio output signal and, as a consequence, also theacoustic output signal toward a desired behavior. Thereby, the desiredbehavior may be characterized by a clear perceptibility of the acousticoutput signal in case of a noisy environment and by a rather gentlelevel of the acoustic output signal in case of a comparatively silentacoustic environment.

According to a further aspect of the invention there is provided anaudio device comprising (a) an acoustic output device for outputting anacoustic output signal based in response to an audio output signal, (b)an acoustic input device for providing an audio input signal in responseto a feedback portion of the acoustic output signal and/or in responseto a current acoustic environmental condition, and (c) a data processor.The data processor is adapted to control any embodiment of the abovedescribed audio device behavior adaptation method.

The described audio device may be a mobile communication end device suchas for instance a mobile phone. The acoustic output device may be forinstance a loudspeaker. The acoustic input device may be for instance amicrophone.

According to a further embodiment of the invention the audio devicefurther comprises a sensor device, which is coupled to the dataprocessor. Thereby, the data processor is adapted to take into account asensor signal generated by the sensor device for adapting the audiooutput signal.

The described sensor device may be capable of detecting any measurablevariable of the audio device and/or of the environment of the audiodevice. By taking into account also the sensor signal the audio outputsignal can be adapted even more precise towards a desired behaviordepending on the environmental acoustic conditions.

The sensor device may comprise any type of sensor such as for instance alight sensitive sensor, a motion sensor, an acceleration sensor and/or aproximity sensor.

According to a further aspect of the invention there is provided acomputer-readable medium on which there is stored a computer program forcontrolling a behavior of an audio device to a current acousticenvironmental condition. The computer program, when being executed by adata processor, is adapted for controlling any embodiment of the abovedescribed audio output signal adaptation method.

According to a further aspect of the invention there is provided aprogram element for controlling an adaptation of a behavior of an audiodevice to a current acoustic environmental condition. The programelement, when being executed by a data processor, is adapted forcontrolling any embodiment of the above described audio output signaladaptation method.

The program element may be implemented as computer readable instructioncode in any suitable programming language, such as, for example, JAVA,C++, and may be stored on a computer-readable medium (removable disk,volatile or non-volatile memory, embedded memory/processor, etc.). Theinstruction code is operable to program a computer or any otherprogrammable device to carry out the intended functions. The programelement may be available from a network, such as the World Wide Web,from which it may be downloaded.

The invention may be realized by means of a computer programrespectively software. However, the invention may also be realized bymeans of one or more specific electronic circuits respectively hardware.Furthermore, the invention may also be realized in a hybrid form, i.e.in a combination of software modules and hardware modules.

It has to be noted that embodiments of the invention have been describedwith reference to different subject matters. In particular, someembodiments have been described with reference to method type claimswhereas other embodiments have been described with reference toapparatus type claims. However, a person skilled in the art will gatherfrom the above and the following description that, unless othernotified, in addition to any combination of features belonging to onetype of subject matter also any combination between features relating todifferent subject matters, in particular between features of the methodtype claims and features of the apparatus type claims is considered asto be disclosed with this application.

The aspects defined above and further aspects of the present inventionare apparent from the examples of embodiment to be described hereinafterand are explained with reference to the examples of embodiment. Theinvention will be described in more detail hereinafter with reference toexamples of embodiment but to which the invention is not limited.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows in accordance with the invention an audio device, whichcomprises an adaptive filter for determining the relation between theaudio output signal x′(t) and the audio input signal z(t).

FIG. 2 shows a block diagram indicating the operation of the audiodevice depicted in FIG. 1.

DESCRIPTION OF EMBODIMENTS

The illustration in the drawing is schematically. It is noted that indifferent Figures, similar or identical elements are provided withreference signs, which are different from the corresponding referencesigns only within the first digit.

FIG. 1 shows an audio device 100 in accordance with the invention.According to the embodiment described here the audio device is a mobilephone 100. The mobile phone comprises an acoustic output device 110 andan acoustic input device 120. The acoustic output device is aloudspeaker 110, the acoustic input device is a microphone 120. Theloudspeaker 110 is driven by an audio output signal x′(t). The audiooutput signal x′(t) is generated by a loudness enhancement unit 111. Theaudio output signal x′(t) is generated based on an original audio signalx(t), which is fed to the loudness enhancement unit 111. According tothe embodiment described here the original audio signal x(t) and theaudio output signal x′(t) represent a ringtone for the mobile phone 100.The ringtone may in particular indicate an incoming call.

Due to a relatively close distance between the loudspeaker 110 and themicrophone 120 there will be a strong acoustic coupling between theloudspeaker 110 and the microphone 120. As a consequence, a feedbacksignal, which is a portion of the acoustic output signal generated bythe loudspeaker, will propagate from the loudspeaker 110 to themicrophone 120. The strength of this coupling depends on the acousticproperty of the mobile phone 100 and of the environment of the mobilephone 100. If the mobile phone 100 is located for instance in a pocketor a bag, the acoustic coupling may be attenuated. Further, thefrequency distribution of the received feedback signal and the acousticoutput signal may be different because of a frequency dependentattenuation.

As can be seen from FIG. 1, the mobile phone further comprises anadaptive filter 112. The adaptive filter 112 receives the audio outputsignal x′(t). The adaptive filter 112 is connected with an adding unit122, which receives an estimated feedback signal y(t) from the adaptivefilter 112. Further, the adaptive filter 112 is connected with ananalysis and control unit 114, which also receives the estimatedfeedback signal y(t). This means that the adaptive filter 112 emulatesthe acoustic path between the audio output signal x′(t) and an audioinput signal z(t) generated by the microphone. This acoustic path alsoincludes the acoustic properties of the loudspeaker 110 and of themicrophone 120.

The audio input signal z(t) is indicative for the acoustical inputsignal captured by the microphone 120. This acoustical input signal isthe sum of the feedback signal and an ambient noise signal.

As can be further seen from FIG. 1, the estimated feedback signal y(t)is fed to a negative input of the adding unit 122. A positive input ofthe adding unit 122 is fed with the audio input signal z(t). The addingunit 122 calculates the difference between the audio input signal z(t)and the estimated feedback signal y(t). Therefore, the adding unit 122acts as a subtraction unit. The difference between the audio inputsignal z(t) and the estimated feedback signal y(t) is a residual signalr(t), which contains the sum of the ambient noise and the remainingfeedback signal not modeled by the adaptive filter.

According to the embodiment described here the mobile phone 100 furthercomprises a sensor device 140. The sensor device 140 generates a sensorsignal q(t), which is fed to the analysis and control unit 114.

Descriptive speaking, FIG. 1 depicts an example of a possibleimplementation of the invention applied for ambient ringtone playbacksignal (i.e. the audio output signal x′(t)) of a mobile phone 100. Inthis embodiment the relation between the playback signal x′(t) and thecaptured microphone signal z(t) is estimated using the adaptive filter112. The resulting estimated feedback signal y(t), the residual signalr(t) and the filter coefficients of the adaptive filter 112 are used formeasuring changes and disturbances introduced by the acoustic propertiesof the environment of the mobile phone 100. Details of this embodimentare described in the following with reference to FIG. 2.

FIG. 2 shows a block diagram of the operation of the audio device 100.In the described embodiment the relation between the audio output signalx′(t) representing the playback signal and the audio input signal z(t)representing the captured microphone signal is estimated by means of anadaptive filter. The resulting estimated feedback signal, the residualsignal r(t) and the filter coefficients of the adaptive filter are usedfor measuring changes and disturbances introduced by the acousticproperties of the environment. In the following the operation of eachblock will be described consecutively.

Block 212: Adaptive Filtering

According to the embodiment described here the adaptive filteringprocedure is carried out by means of an acoustic echo canceller adaptivefilter such as for instance a normalized least mean square adaptivefilter. The adaptive filter has as inputs (a) the audio output signalrespectively the ringtone signal x′(t) which is played through theloudspeaker of the mobile phone, and (b) the audio input signalrespectively the captured microphone signal z(t). The adaptive filtermodels the electro-mechanical acoustic echo path between the microphonesignal z(t) and the reference signal x′(t). The outputs of the adaptivefilter are the feedback respectively the echo estimate y(t) and theresidual signal r(t). These outputs are used by the block 212 a and theblock 212 b for a time and frequency analysis to measure the feedbackreduction performance (the ratio between the determined feedback signaland the residual signal) of the adaptive filter to analyze thedisturbance introduced by the ambient noise. The correspondingcoefficients w_(t)[k] of the adaptive filter, which represent theestimated feedback path, are used by block 212 c for monitoring thedynamic behavior of the acoustical feedback path of the mobile phone inits environment. Thereby, k is the number of the respective filtercoefficient.

Block 212 a: Frequency-Domain Analysis

The block 212 a performs a time-to-frequency transformation, e.g. aDiscrete Fourier Transform, on the signal y(t) and r(t) in order toanalysis the frequency content of the signals. Thereby, the respectivesignals Y_(t)(f) and R_(t)(f) are generated. The output signals of thetime-to-frequency transformation are used in block 214 to analyze thefeedback reduction performance of the adaptive filter.

Block 212 b: Time-Domain Analysis

The block 212 b performs a broadband power calculation on the signalx′(t) as described by the following equation (1):

$\begin{matrix}{{P_{x}(t)} = {\sum\limits_{t\; 1}^{t\; 2}\;{x^{\prime}(t)}^{2}}} & (1)\end{matrix}$

This power P_(X)(t) is compared to a threshold P_(x) _(—) _(Threshold)in order to select the desired parts of the ringtone as described inequation (2) for measuring the performance of the adaptive filter.Desired_(—) X_Signal(t)=(P _(x)(t)>P _(x) _(—) _(Threshold))  (2)

Applying equation (2) can be understood as a ringtone power detection.Preferably, the threshold P_(X) _(—) _(Threshold) is to be tuned to theacoustics of the mobile phone.

Block 212 c: Adaptive Filter Coefficient Analysis

This block 212 c performs the analysis on the adaptive filtercoefficient w_(t)[k] to monitor the dynamic behavior of the acousticalfeedback path of the mobile phone in its environment. Two differentmeasures can be calculated:

-   A) The normalized Euclidian distance Δ_(w)(t) of the filter    coefficient over time, calculated according to equation (3).

$\begin{matrix}{{\Delta_{w}(t)} = \sqrt{\left( \frac{\sum\limits_{0}^{N}\;\left( {{w_{t}\lbrack k\rbrack} - {w_{t - 1}\lbrack k\rbrack}} \right)^{2}}{\sum\limits_{0}^{N}\;\left( {w_{t}\lbrack k\rbrack} \right)^{2}} \right)}} & (3)\end{matrix}$

-   B) The sum of the filter coefficients Sum Coeff(t), calculated    according to equation (4). Thereby the state of the adaptive filter    is calculated.

$\begin{matrix}{{{SumCoeff}(t)} = {\sum\limits_{0}^{N}\;\left( {w_{t}\lbrack k\rbrack} \right)^{2}}} & (4)\end{matrix}$

The value of the normalized Euclidian distance Δ_(w)(t) is low if themobile phone is in a steady state. If the value of Δ_(w)(t) is higherthan a certain threshold Δ_(Threshold), this means that the adaptivefilter is adapting to a new environment. This change of environment iscalled a path change, for example caused by a hand being near the mobilephone, or the mobile phone being moved from an initial location to a newlocation, etc.

By means of the following equation (5) a divergence of the adaptivefilter can be detected.AdaptiveFilterDiverged(t)=(Δ_(w)(t)>Δ_(Threshold))  (5)

Initially the adaptive filter needs to adapt to the environment. Theconvergence of the adaptive filter can be detected by applying thefollowing equation (6):AdaptiveFilterConverged(t)=(Δ_(w)(t)<Δ_(Threshold))  (6)

The value of Sum Coeff(t) is compared to a reference value SumCoeff_(Reference).

This reference value represents the acoustical coupling when the mobilephone is lying in an open environment on a desk. The reference value SumCoeff_(Reference) threshold depends on the acoustics of the device.

If the value Sum Coeff(t) differs by more than a certain percentageΔ_(Sum Coeff) compared to the reference value Sum Coeff_(Reference), itcan be assumed that the mobile phone is located in a closed environmentcausing the acoustical coupling to be higher or lower. According to theembodiment described here this check is done for two different timeintervals, an initial time period [T_(SumCoeff1): T_(Sum Coeff 2)] afterconvergence of the adaptive filter and the consecutive time period[T_(SumCoeff2): ∞] as shown in the following equation (7) and thefollowing equation (8). The value T_(SumCoeff1) is equal to the momentin that the adaptive filter has initially converged (value ofAdaptiveFilterConverged(t) changing from 0 to 1). In other words,equation (7) represents a detector for the initial acoustical couplingstate of the adaptive filter and equation (8) represents a detector forthe modified acoustical coupling state of the adaptive filter.

$\begin{matrix}{{\forall{t \in \left\lbrack {T_{{SumCoeff}\; 1}\;\text{:}T_{{SumCoeff}\; 2}} \right\rbrack}},} & (7) \\{{{AdaptiveFilterInitialState}\mspace{14mu}(t)} = \left\{ \begin{matrix}0 & \left( {{{SumCoeff}(t)} < {\left( {1 - \Delta_{SumCoeff}} \right) \times {SumCoeff}_{Reference}}} \right) \\2 & \left( {{{SumCoeff}(t)} > {\left( {1 + \Delta_{SumCoeff}} \right) \times {SumCoeff}_{Reference}}} \right) \\1 & {otherwise}\end{matrix} \right.} & \; \\{\forall{t \in \left\lbrack {T_{{SumCoeff}\; 2}\;\text{:}{\infty\left\lbrack , \right.}} \right.}} & (8) \\{{{AdaptiveFilterModifiedState}\mspace{14mu}(t)} = \left\{ \begin{matrix}0 & \left( {{{SumCoeff}(t)} < {\left( {1 - \Delta_{SumCoeff}} \right) \times {SumCoeff}_{Reference}}} \right) \\2 & \left( {{{SumCoeff}(t)} > {\left( {1 + \Delta_{SumCoeff}} \right) \times {SumCoeff}_{Reference}}} \right) \\1 & {otherwise}\end{matrix} \right.} & \;\end{matrix}$Block 214: Ratio Calculation and Verification

The raw performance RatioEcho(t)) of the adaptive filter is measured bycomparing the power of Y_(t)(f) and R_(t)(f) for certain frequency bins.Equation (9) is used to calculate the performance of the adaptivefilter.

$\begin{matrix}{{{RatioEcho}(t)} = \left( \frac{\sum\limits_{f\; 1}^{f\; 2}\;{Y_{t}(f)}^{2}}{\sum\limits_{f\; 1}^{f\; 2}\;{R_{t}(f)}^{2}} \right)} & (9)\end{matrix}$

A “filtered” performance RatioEchoFilt(t) of the adaptive filter iscalculated depending on the positive detection in block 212 b accordingto equation (2).

This “filtered” performance RatioEchoFilt(t) of the adaptive filter iscompared to a performance threshold RatioEcho_(Threshold) as describedin equation (10). This threshold depends on the acoustics of the mobilephone and on the desired amount volume increase to be applied to x(t).In other words, equation (10) is used to detect a poor performance ofthe adaptive filter.PoorPerformance(t)=(RatioEchoFilt<RatioEcho_(Threshold))  (10)

The results of this detector are used to calculate an adequate volumechange in block 211. The performance can be calculated for severalfrequency bands in order to obtain more information about theperformance of the adaptive filter in the different frequency bands.This information can then be used to equalize the signal x(t) to enhancethe audibility in the noisy environment.

Block 211: Gain and Frequency Calculation and Application

This block 211 calculates the gain, compression and/or the equalization,basically any filtering that needs to be applied to the original audiosignal x(t) to enhance the loudness of ringtone with respect to itsenvironment. This calculation depends on the detection results of block214 and block 212 c. The following describes a possible gain functionimplementation:

For every period of time T_(GainAnalysis) after convergence of theadaptive filter (equation 6), if a poor performance of the adaptivefilter has been detected by the block 212 c (equation 10), the gain willbe increased with a certain value G_(Increase).

The value G_(Increase) is depending on the value ofAdaptiveFilterInitialState(t). If the valueAdaptiveFilterInitialState(t) is equal to 0, indicating that theringtone playback is muffled, a higher increase value G_(IncreaseHigh)is used.

If the value of AdaptiveFilterInitialState(t) orAdaptiveFilterModifiedState(t) is equal to 2 after T_(SumCoeff2), themobile phone is assumed to be located in a closed environment. In thiscase the gain is increased to a certain high gain value to compensatethe fact that the ringtone is muffled as well.

If a path change or a change in state of the adaptive filter has beendetected, the gain increase is stopped for a certain period of time.Furthermore, if the value of AdaptiveFilterModifiedState(t) is differentfrom 1 after T_(SumCoeff2), this indicates that the mobile phone hasbeen picked-up by a user. In this case, the gain is lowered to itsinitial value.

Block 212: Sensor Data Analysis

This block 212, which is optional, performs an analysis of other sensordata provided by a sensor signal q(t) in order to give additionalinformation about the environment of the device, which can enhance thedetection. As has already been mentioned above, the sensor signal q(t)may be provided by any context sensor, which is capable of detecting ameasurable variable of the mobile phone and/or of the environment of themobile phone. The additional consideration of the sensor signal q(t) mayprovide the advantage, that the audio output signal can be adapted veryprecisely towards its desired behavior depending on the environmentalacoustic conditions.

The sensor providing the sensor signal q(t) may be a light sensitivesensor, a motion sensor, an acceleration sensor and/or a proximitysensor. Preferably, the sensor is a built-in sensor of the mobile phone.

Apart from the mobile phone application of the invention described abovethe audio output signal adaptation procedure described in thisapplication may also be used for other applications. The describedacoustical monitor and detection mechanism based on analyzing thedynamics in a determined relation between the audio playback signal andthe captured microphone signal can generally be used to steer anyplayback audio device and its playback towards a desired behavior.Specifically, the described audio output signal adaptation can be usedfor instance for an automatic ambient noise adaptive speech enhancement.Further, an automatic ambient noise adaptive playback on any audiodevices may be implemented, on which direct noise level measurementsfrom the captured microphone signal are not possible because the ambientnoise is masked by the echo from the audio playback. Furthermore, thedescribed mechanism can be used for an acoustical detector using theloudspeaker and the microphone signal to steer the audio device and theaudio playback towards its desired behavior in response to a change inor detection of a certain acoustical environment of the audio device,e.g. a proximity detector.

It should be noted that the term “comprising” does not exclude otherelements or steps and “a” or “an” does not exclude a plurality. Alsoelements described in association with different embodiments may becombined. It should also be noted that reference signs in the claimsshould not be construed as limiting the scope of the claims.

In order to recapitulate the above described embodiment of the presentinvention one can state:

In this application there is described an acoustical monitor anddetection system to steer the adaptation or enhancement of acousticoutput signals of an audio device depending on the acousticcharacteristics of the environment of the audio device includingadaptation of other functionality of the audio device. The audio devicecomprises an acoustic output device such as for instance a loudspeakerand an acoustic input device such as for instance a microphone. Theacoustic properties of the environment influence a relation or a mappingbetween the audio output signal producing the acoustic output signal andthe audio input signal being captured by the acoustic input device. Achange or a disturbance in the environment of the audio device causes achange or a disturbance in a determined or estimated relation betweenthe audio output signal and the captured audio input signal. Bymeasuring and monitoring this relation or derivative of these signalsand its dynamics, the audio device can identify changes or disturbancesin the environment of the with respect to reference situations. Thereby,an acoustical detection mechanism is defined, which is used to steer theaudio device and in particular the audio output signal towards a desiredbehavior depending on the acoustic environmental conditions.

More specific, this invention allows adaptation of ringtone or audioplayback on mobile audio devices depending on the level of theenvironmental background noise, which is not possible by direct noisemeasurement techniques due to a high acoustical coupling between theacoustic output device and the acoustic input device. In addition, byevaluating the above described relation between the audio output signaland the captured audio input signal a detection mechanism can beestablished, which can find out whether the mobile audio device iscovered or is located in a closed environment like a pocket or a bagduring a ringtone playback representing the above mentioned acousticoutput signal. This situation requires as well an adjustment of theringtone volume and equalization accordingly, so that the ringtone canbe heard. In addition, by evaluating the above described relationbetween the audio output signal and the captured audio input signal anacoustic detection mechanism may be provided for detecting a pick-up ofthe mobile audio device so that the ringtone playback level can bereduced back to a soft, comfortable level or mute when answering thecall has already started.

REFERENCE NUMERALS

-   100 audio device/mobile phone-   110 acoustic output device/loudspeaker-   111 loudness enhancement unit-   112 adaptive filter-   114 analysis and control unit-   120 acoustic input device/microphone-   122 adding unit-   140 sensor device-   r(t) residual signal-   q(t) sensor signal-   x(t) original audio signal-   x′(t) audio output signal-   y(t) estimated feedback signal-   z(t) audio input signal-   w_(t)[k] adaptive filter coefficients-   211 Gain and Frequency Calculation and application-   212 Adaptive Filtering-   212 a Frequency Analysis-   212 b Time Analysis-   212 c Adaptive Filter Coefficients Analysis-   214 Ratio Calculation and Verification-   242 Sensor Data Analysis-   r(t) residual signal-   R_(t)(f) fourier transform of r(t)-   q(t) sensor signal-   x(t) original audio signal-   x′(t) audio output signal-   y(t) estimated feedback signal-   Y_(t)(f) fourier transform of y(t)-   z(t) audio input signal

The invention claimed is:
 1. A method for controlling an adaptation of abehavior of an audio device to a current acoustic environmentalcondition, the method comprising: monitoring an audio output signalbeing provided to an acoustic output device of the audio device foroutputting an acoustic output signal, measuring an audio input signalbeing provided by an acoustic input device of the audio device, whereinthe audio input signal is indicative for a feedback portion of theacoustic output signal and for the current acoustic environmentalcondition, receiving the audio signal by an adaptive filter, providingan estimated feedback signal from the adaptive filter, determining aresidual signal as a difference between the audio input signal and theestimated feedback signal, performing a time-to-frequency transformationon the estimated feedback signal and on the residual signal to generatethe respective time-to-frequency transformation output signals and,using the time-to-frequency transformation output signals to analyze thefeedback reduction performance of the adaptive filter by comparing thepower of the time-to-frequency transformation output signals for certainfrequency bins, determining a relation between the audio output signaland the audio input signal from the analyzed feedback reductionperformance of the adaptive filter, and adapting the behavior of theaudio device based on the determined relation.
 2. The method as setforth in claim 1, wherein the behavior of the audio device is given byan amplitude and/or a frequency of the audio output signal, an amplitudeand/or a frequency of a vibrating mechanism of the audio device and/or amodification of the operation of a display of the audio device.
 3. Themethod as set forth in claim 1, wherein the relation between the audiooutput signal and the audio input signal is determined by applying across-correlation procedure, an adaptive filtering procedure and/or acoherence estimation procedure.
 4. The method as set forth in claim 1,wherein the acoustic output device is a loudspeaker and/or the acousticinput device is a microphone.
 5. The method as set forth in claim 1,further comprising: comparing the determined relation between the audiooutput signal and the audio input signal with at least one referencerelation, wherein adapting the behavior of the audio device furthertakes into account a result of the comparison between the determinedrelation and the reference relation.
 6. The method as set forth in claim1, further comprising: comparing the determined relation between theaudio output signal and the audio input signal with a threshold, whereinif the determined relation is larger or smaller than the threshold,adapting the behavior of the audio device comprises increasing thesignal level of the audio output signal.
 7. The method as set forth inclaim 1, wherein the audio device is a mobile communication end device.8. The method as set forth in claim 7, further comprising: detecting apicking up of the mobile communication end device based on a change inthe determined relation between the audio output signal and the audioinput signal.
 9. The method as set forth in claim 1, further comprising:generating a sensor signal by a sensor device, wherein adapting theaudio output signal further takes into account the sensor signal. 10.The method as set forth in claim 9, wherein the sensor device comprises:a light sensitive sensor, a motion sensor, an acceleration sensor,and/or a proximity sensor.
 11. A data processor for controlling anadaptation of a behavior of an audio device to a current acousticenvironmental condition of the audio device, wherein the data processoris adapted for performing the method as set forth in claim
 1. 12. Anaudio device comprising: an acoustic output device for outputting anacoustic output signal based in response to an audio output signal, anacoustic input device for providing an audio input signal in response toa feedback portion of the acoustic output signal and/or in response to acurrent acoustic environmental condition, and a data processor as setforth in claim
 11. 13. The audio device as set forth in claim 12,further comprising: a sensor device, which is coupled to the dataprocessor, wherein the data processor is adapted to take into account asensor signal generated by the sensor device for adapting the audiooutput signal.
 14. A non-transitory computer-readable medium on whichthere is stored a computer program for controlling an adaptation of abehavior of an audio device to a current acoustic environmentalcondition, the computer program, when being executed by a dataprocessor, is adapted for monitoring an audio output signal beingprovided to an acoustic output device of the audio device for outputtingan acoustic output signal, measuring an audio input signal beingprovided by an acoustic input device of the audio device, wherein theaudio input signal is indicative for a feedback portion of the acousticoutput signal and for the current acoustic environmental condition,receiving the audio signal by an adaptive filter, providing an estimatedfeedback signal from the adaptive filter, determining a residual signalas a difference between the audio input signal and the estimatedfeedback signal, performing a time-to-frequency transformation on theestimated feedback signal and on the residual signal to generate therespective time-to-frequency transformation output signals and, usingthe time-to-frequency transformation output signals to analyze thefeedback reduction performance of the adaptive filter by comparing thepower of the time-to-frequency transformation output signals for certainfrequency bins, determining a relation between the audio output signaland the audio input signal from the analyzed feedback reductionperformance of the adaptive filter, and adapting the behavior of theaudio device based on the determined relation.